MELOGRAPH: Multi-Engine WorkfLOw Graph Processing
نویسنده
چکیده
This paper introduces MELOGRAPH, a new system that exposes in the front-end a domain specific language(DSL) for graph processing tasks and in the back-end identifies, ranks and generates source code for the top-N ranked engines. This approach lets the specialized MELOGRAPH be part of a more general multi-engine workflow optimizer. The candidate execution engines are chosen from the contemporaneous Big Data ecosystem: graph databases (e.g. Neo4j, TitanDB, OrientDB, Sparksee/DEX) and robust graph processing frameworks with Java API or packaged libraries of algorithms (e.g. Giraph, Okapi, Flink Gelly, Hama, Gremlin). As MELOGRAPH is work in progress, our current paper stresses upon the state of the art in this field, provides a general architecture and some early implementation insights.
منابع مشابه
Workflow Engine with Multi-Level Parallelism Supports
This paper presents the SWFL workflow engine, a general workflow framework that meets the needs of business processes as well as scientific computing processes with fine multi-level parallelism supports. The workflow description language, SWFL, follows a graph-oriented model to specify workflow processes composed of services. The workflow engine provides an efficient enactment environment for S...
متن کاملAn Adaptive Workflow Engine Based on Web Services and Agents
At present workflow management systems are widely used to manage business processes in an organization, while the workflow engine provides a runtime execution environment, manages the overall processing and execution of workflow instances. One of the major limitations of current workflow engine is the lack of flexibility to support adaptive environments. In this paper we propose an adaptive wor...
متن کاملBiobankCloud: A Platform for the Secure Storage, Sharing, and Processing of Large Biomedical Data Sets
Biobanks store and catalog human biological material that is increasingly being digitized using next-generation sequencing (NGS). There is, however, a computational bottleneck, as existing software systems are not scalable and secure enough to store and process the incoming wave of genomic data from NGS machines. In the BiobankCloud project, we are building a Hadoop-based platform for the secur...
متن کاملBuilding a Customizable Business-Process-as-a-Service Application with Current State-of-Practice
Application-level multi-tenancy is an increasingly prominent architectural pattern in Software-as-a-Service (SaaS) applications that enables multiple tenants (customers) to share common application functionality and resources among each other. This has the disadvantage that multi-tenant applications are often limited in terms of customizability: one application should fit the needs of all custo...
متن کاملArchitectural Plan for Constructing Fault Tolerable Workflow Engines Based on Grid Service
In this paper the design and implementation of fault tolerable architecture for scientific workflow engines is presented. The engines are assumed to be implemented as composite web services. Current architectures for workflow engines do not make any considerations for substituting faulty web services with correct ones at run time. The difficulty is to rollback the execution state of the workflo...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2016